AI tools for Athina Evals Python SDK
Related Tools:
Athina AI
Athina AI is a comprehensive platform designed to monitor, debug, analyze, and improve the performance of Large Language Models (LLMs) in production environments. It provides a suite of tools and features that enable users to detect and fix hallucinations, evaluate output quality, analyze usage patterns, and optimize prompt management. Athina AI supports integration with various LLMs and offers a range of evaluation metrics, including context relevancy, harmfulness, summarization accuracy, and custom evaluations. It also provides a self-hosted solution for complete privacy and control, a GraphQL API for programmatic access to logs and evaluations, and support for multiple users and teams. Athina AI's mission is to empower organizations to harness the full potential of LLMs by ensuring their reliability, accuracy, and alignment with business objectives.
Athina AI
Athina AI is a platform that provides research and guides for building safe and reliable AI products. It helps thousands of AI engineers in building safer products by offering tutorials, research papers, and evaluation techniques related to large language models. The platform focuses on safety, prompt engineering, hallucinations, and evaluation of AI models.
Athina AI Hub
Athina AI Hub is an ultimate resource for AI development teams, offering a wide range of AI development blogs, research papers, and original content. It provides valuable insights into cutting-edge technologies such as Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and AI agents. Athina AI Hub aims to empower AI engineers, researchers, data scientists, and product developers by offering comprehensive resources and fostering innovation in the field of Artificial Intelligence.
Athena Intelligence
Athena Intelligence is an AI-native analytics platform and artificial employee designed to accelerate analytics workflows by offering enterprise teams co-pilot and auto-pilot modes. Athena learns your workflow as a co-pilot, allowing you to hand over controls to her for autonomous execution with confidence. With Athena, everyone in your enterprise has access to a data analyst, and she doesn't take days off. Simple integration to your Enterprise Data Warehouse Chat with Athena to query data, generate visualizations, analyze enterprise data and codify workflows. Athena's AI learns from existing documentation, data and analyses, allowing teams to focus on creating new insights. Athena as a platform can be used collaboratively with co-workers or Athena, with over 100 users in the same report or whiteboard environment concurrently making edits. From simple queries and visualizations to complex industry specific workflows, Athena enables you with SQL and Python-based execution environments.
Vanna.AI
Vanna.AI is a personalized AI SQL agent that allows users to get actionable insights from their database by simply asking questions. It helps users to write SQL queries effortlessly, even if they are not SQL experts. Vanna.AI is open-source, highly accurate on complex datasets, designed for security, self-learning, and supports various databases. It is fully customizable to match unique needs and offers different front-end integrations for user convenience.
Athena AI
Athena AI is an enterprise chat tool powered by generative AI that enables teams to communicate with their data and receive accurate responses. It connects to various data sources, including files, databases, and third-party applications, allowing users to ask questions, extract insights, and make informed decisions. Athena AI aims to enhance productivity, streamline workflows, and improve collaboration within organizations.
ConverSight
ConverSight is an AI-powered augmented analytics tool designed to empower business leaders with generative AI capabilities. It offers effortless data exploration, automated insights, and workflows, as well as data consolidation and streamlined processes. With over 100+ integrations, ConverSight enables users to centralize data, collaborate with team members, and accurately plan for the future. The platform is a low-code solution that provides self-creating dashboards, guided recommendations, and conversational analytics, making it a valuable tool for decision-making and data analysis in various industries and roles.
SQLAI.ai
SQLAI.ai is a powerful SQL multi-tool that utilizes AI to generate, fix, explain, and optimize SQL queries and databases. It empowers users to effortlessly create complex SQL queries using everyday language, optimize queries for better performance, fix syntax errors with ease, and gain a deeper understanding of SQL queries through AI-powered explanations. Additionally, SQLAI.ai enables users to train AI on their database schema, ensuring unparalleled accuracy in AI-generated queries and optimizations.
CounselorGPT - AI Therapist and Psychologist
Athena AI provides empathetic counseling and general advice to help you navigate life’s many challenges.
Oraculum
Create, Edit or Replicate images! Pro Settings. Updated 12/24 🎄 v0.5. ~~~~Oraculum embodies the visionary spirit of Delphi’s ancient seers, crafting precise AI media with the wisdom of Hephaestus’ forge and the grace of Athena’s olive branch. Show or speak your vision.
Athena
I´m the Representative and personal Assistant of KenoNitroProject (KNP), an adventorous Endevour that bursts the Bounds of Imagination. Let´s talk about Philosophy, Science, Technology, SciFi and Mythology - just about Everything ;)
Athena Notes AI
I convert transcripts into detailed meeting notes with insights, summaries, and action items, plus a downloadable MS Word file.
athina-evals
Athina is an open-source library designed to help engineers improve the reliability and performance of Large Language Models (LLMs) through eval-driven development. It offers plug-and-play preset evals for catching and preventing bad outputs, measuring model performance, running experiments, A/B testing models, detecting regressions, and monitoring production data. Athina provides a solution to the flaws in current LLM developer workflows by offering rapid experimentation, customizable evaluators, integrated dashboard, consistent metrics, historical record tracking, and easy setup. It includes preset evaluators for RAG applications and summarization accuracy, as well as the ability to write custom evals. Athina's evals can run on both development and production environments, providing consistent metrics and removing the need for manual infrastructure setup.
Awesome-LLM-Eval
Awesome-LLM-Eval: a curated list of tools, benchmarks, demos, papers for Large Language Models (like ChatGPT, LLaMA, GLM, Baichuan, etc) Evaluation on Language capabilities, Knowledge, Reasoning, Fairness and Safety.
rag-cookbooks
Welcome to the comprehensive collection of advanced + agentic Retrieval-Augmented Generation (RAG) techniques. This repository covers the most effective advanced + agentic RAG techniques with clear implementations and explanations. It aims to provide a helpful resource for researchers and developers looking to use advanced RAG techniques in their projects, offering ready-to-use implementations and guidance on evaluation methods. The RAG framework addresses limitations of Large Language Models by using external documents for in-context learning, ensuring contextually relevant and accurate responses. The repository includes detailed descriptions of various RAG techniques, tools used, and implementation guidance for each technique.
genai-quickstart-pocs
This repository contains sample code demonstrating various use cases leveraging Amazon Bedrock and Generative AI. Each sample is a separate project with its own directory, and includes a basic Streamlit frontend to help users quickly set up a proof of concept.
litellm
LiteLLM is a tool that allows you to call all LLM APIs using the OpenAI format. This includes Bedrock, Huggingface, VertexAI, TogetherAI, Azure, OpenAI, and more. LiteLLM manages translating inputs to provider's `completion`, `embedding`, and `image_generation` endpoints, providing consistent output, and retry/fallback logic across multiple deployments. It also supports setting budgets and rate limits per project, api key, and model.
aiges
AIGES is a core component of the Athena Serving Framework, designed as a universal encapsulation tool for AI developers to deploy AI algorithm models and engines quickly. By integrating AIGES, you can deploy AI algorithm models and engines rapidly and host them on the Athena Serving Framework, utilizing supporting auxiliary systems for networking, distribution strategies, data processing, etc. The Athena Serving Framework aims to accelerate the cloud service of AI algorithm models and engines, providing multiple guarantees for cloud service stability through cloud-native architecture. You can efficiently and securely deploy, upgrade, scale, operate, and monitor models and engines without focusing on underlying infrastructure and service-related development, governance, and operations.
YesImBot
YesImBot, also known as Athena, is a Koishi plugin designed to allow large AI models to participate in group chat discussions. It offers easy customization of the bot's name, personality, emotions, and other messages. The plugin supports load balancing multiple API interfaces for large models, provides immersive context awareness, blocks potentially harmful messages, and automatically fetches high-quality prompts. Users can adjust various settings for the bot and customize system prompt words. The ultimate goal is to seamlessly integrate the bot into group chats without detection, with ongoing improvements and features like message recognition, emoji sending, multimodal image support, and more.
vircadia-native-core
Vircadia™ is an open source agent-based metaverse ecosystem that excels in mass human and agent (AI) based immersive worlds. It offers mobile, desktop, and VR support through the web, allows hundreds of agents simultaneously, supports full-body (human or agents), scripting with JavaScript & TypeScript, visual scripting, full world editor, 4096km³ world space in a server, fully self-hosted, and more. Vircadia is sponsored by various companies, organizations, and governments. An 'agent' in Vircadia is an AI being that shares the same space as users, interacting, speaking, and experiencing the world, used for companionship, training, and gameplay opportunities. Vircadia excels at deploying agents en-masse for a full sandbox experience.
aioboto3
aioboto3 is an async AWS SDK for Python that allows users to use near enough all of the boto3 client commands in an async manner just by prefixing the command with `await`. It combines the great work of boto3 and aiobotocore, enabling users to use higher level APIs provided by boto3 in an asynchronous manner. The package provides support for various AWS services such as DynamoDB, S3, Kinesis, SSM Parameter Store, and Athena. It also offers features like client-side encryption using KMS-Managed master keys and supports asyncifying `get_presigned_url`. The library closely mimics the usage of boto3 and is mainly developed to be used in async microservices.
aiobotocore
aiobotocore is an async client for Amazon services using botocore and aiohttp/asyncio. It provides a mostly full-featured asynchronous version of botocore, allowing users to interact with various AWS services asynchronously. The library supports operations such as uploading objects to S3, getting object properties, listing objects, and deleting objects. It also offers context manager examples for managing resources efficiently. aiobotocore supports multiple AWS services like S3, DynamoDB, SNS, SQS, CloudFormation, and Kinesis, with basic methods tested for each service. Users can run tests using moto for mocked tests or against personal Amazon keys. Additionally, the tool enables type checking and code completion for better development experience.
cube
Cube is a semantic layer for building data applications, helping data engineers and application developers access data from modern data stores, organize it into consistent definitions, and deliver it to every application. It works with SQL-enabled data sources, providing sub-second latency and high concurrency for API requests. Cube addresses SQL code organization, performance, and access control issues in data applications, enabling efficient data modeling, access control, and performance optimizations for various tools like embedded analytics, dashboarding, reporting, and data notebooks.
blog
这是一个程序员关于 ChatGPT 学习过程的记录，其中包括了 ChatGPT 的使用技巧、相关工具和资源的整理，以及一些个人见解和思考。 **使用技巧** * **充值 OpenAI API**：可以通过 https://beta.openai.com/account/api-keys 进行充值，支持信用卡和 PayPal。 * **使用专梯**：推荐使用稳定的专梯，可以有效提高 ChatGPT 的访问速度和稳定性。 * **使用魔法**：可以通过 https://my.x-air.app:666/#/register?aff=32853 访问 ChatGPT，无需魔法即可访问。 * **下载各种 apk**：可以通过 https://apkcombo.com 下载各种安卓应用的 apk 文件。 * **ChatGPT 官网**：ChatGPT 的官方网站是 https://ai.com。 * **Midjourney**：Midjourney 是一个生成式 AI 图像平台，可以通过 https://midjourney.com 访问。 * **文本转视频**：可以通过 https://www.d-id.com 将文本转换为视频。 * **国内大模型**：国内也有很多大模型，如阿里巴巴的通义千问、百度文心一言、讯飞星火、阿里巴巴通义听悟等。 * **查看 OpenAI 状态**：可以通过 https://status.openai.com/ 查看 OpenAI 的服务状态。 * **Canva 画图**：Canva 是一个在线平面设计平台，可以通过 https://www.canva.cn 进行画图。 **相关工具和资源** * **文字转语音**：可以通过 https://modelscope.cn/models?page=1&tasks=text-to-speech&type=audio 找到文字转语音的模型。 * **可好好玩玩的项目**： * https://github.com/sunner/ChatALL * https://github.com/labring/FastGPT * https://github.com/songquanpeng/one-api * **个人博客**： * https://baoyu.io/ * https://gorden-sun.notion.site/527689cd2b294e60912f040095e803c5?v=4f6cc12006c94f47aee4dc909511aeb5 * **srt 2 lrc 歌词**：可以通过 https://gotranscript.com/subtitle-converter 将 srt 格式的字幕转换为 lrc 格式的歌词。 * **5 种速率限制**：OpenAI API 有 5 种速率限制：RPM（每分钟请求数）、RPD（每天请求数）、TPM（每分钟 tokens 数量）、TPD（每天 tokens 数量）、IPM（每分钟图像数量）。 * **扣子平台**：coze.cn 是一个扣子平台，可以提供各种扣子。 * **通过云函数免费使用 GPT-3.5**：可以通过 https://juejin.cn/post/7353849549540589587 免费使用 GPT-3.5。 * **不蒜子 统计网页基数**：可以通过 https://busuanzi.ibruce.info/ 统计网页的基数。 * **视频总结和翻译网页**：可以通过 https://glarity.app/zh-CN 总结和翻译视频。 * **视频翻译和配音工具**：可以通过 https://github.com/jianchang512/pyvideotrans 翻译和配音视频。 * **文字生成音频**：可以通过 https://www.cnblogs.com/jijunjian/p/18118366 将文字生成音频。 * **memo ai**：memo.ac 是一个多模态 AI 平台，可以将视频链接、播客链接、本地音视频转换为文字，支持多语言转录后翻译，还可以将文字转换为新的音频。 * **视频总结工具**：可以通过 https://summarize.ing/ 总结视频。 * **可每天免费玩玩**：可以通过 https://www.perplexity.ai/ 每天免费玩玩。 * **Suno.ai**：Suno.ai 是一个 AI 语言模型，可以通过 https://bibigpt.co/ 访问。 * **CapCut**：CapCut 是一个视频编辑软件，可以通过 https://www.capcut.cn/ 下载。 * **Valla.ai**：Valla.ai 是一个多模态 AI 模型，可以通过 https://www.valla.ai/ 访问。 * **Viggle.ai**：Viggle.ai 是一个 AI 视频生成平台，可以通过 https://viggle.ai 访问。 * **使用免费的 GPU 部署文生图大模型**：可以通过 https://www.cnblogs.com/xuxiaona/p/18088404 部署文生图大模型。 * **语音转文字**：可以通过 https://speech.microsoft.com/portal 将语音转换为文字。 * **投资界的 ai**：可以通过 https://reportify.cc/ 了解投资界的 ai。 * **抓取小视频 app 的各种信息**：可以通过 https://github.com/NanmiCoder/MediaCrawler 抓取小视频 app 的各种信息。 * **马斯克 Grok1 开源**：马斯克的 Grok1 模型已经开源，可以通过 https://github.com/xai-org/grok-1 访问。 * **ChatALL**：ChatALL 是一个跨端支持的聊天机器人，可以通过 https://github.com/sunner/ChatALL 访问。 * **零一万物**：零一万物是一个 AI 平台，可以通过 https://www.01.ai/cn 访问。 * **智普**：智普是一个 AI 语言模型，可以通过 https://chatglm.cn/ 访问。 * **memo ai 下载**：可以通过 https://memo.ac/ 下载 memo ai。 * **ffmpeg 学习**：可以通过 https://www.ruanyifeng.com/blog/2020/01/ffmpeg.html 学习 ffmpeg。 * **自动生成文章小工具**：可以通过 https://www.cognition-labs.com/blog 生成文章。 * **简易商城**：可以通过 https://www.cnblogs.com/whuanle/p/18086537 搭建简易商城。 * **物联网**：可以通过 https://www.cnblogs.com/xuxiaona/p/18088404 学习物联网。 * **自定义表单、自定义列表、自定义上传和下载、自定义流程、自定义报表**：可以通过 https://www.cnblogs.com/whuanle/p/18086537 实现自定义表单、自定义列表、自定义上传和下载、自定义流程、自定义报表。 **个人见解和思考** * ChatGPT 是一个强大的工具，可以用来提高工作效率和创造力。 * ChatGPT 的使用门槛较低，即使是非技术人员也可以轻松上手。 * ChatGPT 的发展速度非常快，未来可能会对各个行业产生深远的影响。 * 我们应该理性看待 ChatGPT，既要看到它的优点，也要意识到它的局限性。 * 我们应该积极探索 ChatGPT 的应用场景，为社会创造价值。
promptulate
**Promptulate** is an AI Agent application development framework crafted by **Cogit Lab** , which offers developers an extremely concise and efficient way to build Agent applications through a Pythonic development paradigm. The core philosophy of Promptulate is to borrow and integrate the wisdom of the open-source community, incorporating the highlights of various development frameworks to lower the barrier to entry and unify the consensus among developers. With Promptulate, you can manipulate components like LLM, Agent, Tool, RAG, etc., with the most succinct code, as most tasks can be easily completed with just a few lines of code. 🚀
llamafile
llamafile is a tool that enables users to distribute and run Large Language Models (LLMs) with a single file. It combines llama.cpp with Cosmopolitan Libc to create a framework that simplifies the complexity of LLMs into a single-file executable called a 'llamafile'. Users can run these executable files locally on most computers without the need for installation, making open LLMs more accessible to developers and end users. llamafile also provides example llamafiles for various LLM models, allowing users to try out different LLMs locally. The tool supports multiple CPU microarchitectures, CPU architectures, and operating systems, making it versatile and easy to use.
bedrock-claude-chat
This repository is a sample chatbot using the Anthropic company's LLM Claude, one of the foundational models provided by Amazon Bedrock for generative AI. It allows users to have basic conversations with the chatbot, personalize it with their own instructions and external knowledge, and analyze usage for each user/bot on the administrator dashboard. The chatbot supports various languages, including English, Japanese, Korean, Chinese, French, German, and Spanish. Deployment is straightforward and can be done via the command line or by using AWS CDK. The architecture is built on AWS managed services, eliminating the need for infrastructure management and ensuring scalability, reliability, and security.
Awesome-LLM-Compression
Awesome LLM compression research papers and tools to accelerate LLM training and inference.
Awesome-Segment-Anything
Awesome-Segment-Anything is a powerful tool for segmenting and extracting information from various types of data. It provides a user-friendly interface to easily define segmentation rules and apply them to text, images, and other data formats. The tool supports both supervised and unsupervised segmentation methods, allowing users to customize the segmentation process based on their specific needs. With its versatile functionality and intuitive design, Awesome-Segment-Anything is ideal for data analysts, researchers, content creators, and anyone looking to efficiently extract valuable insights from complex datasets.
